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Abstract 

One of the major concerns of the Engineering Accreditation Council (EAC) is the need for an effective 
monitoring and evaluation of program outcome domains that can be associated with courses taught under the 
Mechanical Engineering program. However, an effective monitoring method that can determine the results of 
each program outcome using Bloom’s Taxonomy has not yet been established for each course. The purpose of 
this research is to conduct a Big Picture Assessment to achieve Outcome-Based Learning. Big Picture 
Assessment is a comprehensive monitoring tool of courses with studied program outcome domains. The tool 
applies the three main domains of Bloom’s Taxonomy, namely, psychomotor, cognitive, and affective, in its 
monitoring process. Furthermore, the identification of program outcomes for each course is evaluated to meet 
standards set by the EAC. The results of this study will facilitate continuous improvement on existing courses. 
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1. Introduction 

The Faculty of Engineering and the Built Environment (FKAB) of the Universiti Kebangsaan Malaysia (UKM) 
prepare themselves to meet the criteria set by the Enginerring Accreditation Council (EAC), which acts as a 
recognition body of engineering programs in Malaysia. This undertaking is aimed at fulfilling the aspirations of 
UKM in becoming a leading research university with an accredited study program. In 2004, all engineering 
programs in public higher education institutions in Malaysia have begun to implement the Outcome-Based 
Education curriculum (OBE) (Shahrir et al., 2008). In line with the implementation of OBE into the engineering 
program, emphasis on the curriculum and the methods of delivery, assessment, and measurement is very 
important and should always be continued to ensure that the Program Education Objectives and Program 
Outcomes (PO) can be applied in teaching and learning. One of the requirements seriously considered by the 
EAC is an effective assessment of PO domains that can be associated with courses taught in engineering 
programs. (EAC, 2012) 

The purpose of this paper is to conduct the Big Picture Assessment on the mechanical engineering (KM) 
program in UKM and include the findings in the EAC self-assesment report. Big Picture Assessment is a 
comprehensive monitoring tool that can be used to assess all KM courses of study with applied domain POs. The 
results for each course program are assessed to determine if these programs meet the standards set by the EAC to 
facilitate improvements when needed. 

2. Methodology 

Big Picture Rating was developed by Excel software. Mapping between POs and KM courses taken in each 
semester was created. The maximum number of POs used was only 9; before 2010, the number was 12. Table 1 
shows the details and definition of each PO. Meanwhile, the courses were matched based on the existing list in 
the FKAB Undergraduate Handbook 2011-2012. Given that the focus was only on the compulsory courses of the 
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KM programme, courses such as electives, engineering fundamentals, and general education were not 
considered. 

Table 1. Details of POs (source: FKAB Undergraduate Handbook 2011-2012) 

POl Ability to apply knowledge in mathematics, science, and engineering 

P02 Ability to identify, formulates, solve, and improve engineering problems using techniques, 

skills, and modem engineering tools necessary for engineering practice 
P03 Ability to design a component, system, or process to meet desired needs 

P04 Understanding of professional and ethical responsibility based on knowledge in environmental 

and contemporary issues 

P05 Ability to understand and apply in-depth knowledge of one or more areas of specialization 

within mechanical engineering 

P06 Ability to design and conduct experiments, as well as to analyze and interpret data 

P07 Ability to communicate and function effectively in a team 

P08 Recognition of the need for and an ability to engage in lifelong learning 

P09 Knowledgeable in project management, administration, business acumen, and entrepreneurship 


In the matrix assessment, two color indicators, red and yellow, were used. Red represents a high correlation, 
while yellow means moderate correlation. If no color is indicated, the PO and course have a low or almost no 
correlation. All assessment criteria were listed in each color matrix, which describe how the POs were evaluated 
for each course. Table 2 lists the assessment criteria for evaluating the POs-KM courses matrix. 


Table 2. Assessment methods used in the matrix of POs-KM courses 


e 

Exam 

r 

Report 

a 

Task 

d 

Design 

P 

Project 

1 

Laboratory work 

c 

Case study 

s 

Software 

PB 

Problem based learning 

m 

Drawing 

g 

Co-curiculum activity 

n 

Observation 

0 

Oral presentation 

f 

Evaluation (peers) 

h 

Survey 

q 

Evaluation (lecturer) 


Each filled matrix was evaluated according to Bloom’s Taxonomy for each KM course. Introduced in 1956, 
Bloom’s Taxonomy categorizes skills and objectives to be achieved by the students into three major domains, 
namely, cognitive, affective, and psychomotor (Aqilah et al., 2008). Each course offered does have certain course 
outcomes based on the results of the cognitive domain in Bloom’s taxonomy. Initiatives have been taken to 
update all the results of the course based on the cognitive domain of Bloom’s taxonomy for engineering 
programs (Hairi et al., 2008). The cognitive domain includes knowledge, comprehension, and critical thinking 
about a certain topic. Traditional education tends to emphasize the skills in this domain. The cognitive domain 
consists of a number of the learning processes that should be implemented, such as the process of remembering, 
understanding, applying, analyzing, evaluating, and creating. The ordering of the processes is illustrated in 
Figure 1. 
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Figure 1. Cognitive domain 


Skills in the affective domain describe the way people react emotionally and their ability to feel sadness or 
happiness towards other living things. The affective domain has five levels, from the lowest to the highest 
process: receiving, responding, evaluating, organizing, and characterizing (Figure 2). 


A5 - Characterizing by Value 
A4- Organizing and Conceptualizing 


A 


A 


A 


A3 - Evaluating 


A2 - Responding 


A1 - Receiving 



Figure 2. Affective domain 


The third domain is the psychomotor domain, which describes the ability to manipulate something physically. 
The objectives of the psychomotor domain usually focus on change and the development of attitudes and skills. 
The psychomotor domain includes the processes of perception, set, guided response, mechanism, complex overt 
reaction, adaptation, and origination, as shown in Figure 3. 
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Figure 3. Psychomotor domain 


3. Results and Discussion 

The matrix between POs and KM courses that reflect the Big Picture Assessment of this program are displayed 
in one sheet only. However, for this paper, the matrix is divided into two phases, from the 1 st to the 4 th semester, 
and from the 5 th to the 8 th semester. Figure 4 shows the matrix from the 1 st to the 4 th semester. The majority of the 
KM courses emphasized on POl, P02, P05, P06, and P07. Only a few of courses incorporated P03. Based on 
the details of these POs, much emphasis has been made on basic skills in science and engineering, environmental 
issues, and the application of knowledge. These matters are given more focus in most courses, enabling students 
to form a strong foundation during their 1 st and 2 nd years before continuing into their 3 rd and 4 th years of study. In 
terms of Bloom’s Taxonomy, all the courses successfully incorporated all three domains. Most courses 
incorporated C3 to C5 of the cognitive domain. For the psychomotor domain, the majority of courses 
incorporated P3 and P5. Meanwhile, for the affective domains, many courses included A3 and A4. Thus, several 
of these courses dealt with these domains on an intermediate level and rarely incorporated the highest levels 
during the 1 st and 2 nd years of study. For the assessment method adopted, most courses used e (examination), a 
(assignment), p (project), r (report), and 1 (lab). Only certain courses included o (oral presentation) as one of their 
assessment methods. Thus, the majority of courses used more traditional evaluation methods that are commonly 
used in evaluating basic POs and that are less difficult in assessing their students. 
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Figure 4. Big picture for KM from the 1 st to the 4 th semester 


Figure 5 shows the courses taken from the 5 th to the 8 th semester (3 rd and 4 th years). Many courses incorporated 
POl to P08. Thus, 3 rd and 4 th year students were tested on all POs compared with 1 st and 2 nd year students. In 
terms of the cognitive domain, many courses included C5 and C6. For the psychomotor domain P3, P4, P5, and 
P7 were used, and, in the affective domain, A3 to A5. Thus, when the students enter their 3 rd and 4 th years, all of 
the highest levels of all three domains are covered. Furthermore, the results are consistent with previous studies 
in indicating that more than 90% of curricula in institutes of higher learning emphasize cognitive skills over 
psychomotor ones (Maizura et al. 2008). Furthermore, during this period, students have to undergo a more 
challenging evaluation in the form of PBL and case studies. Even an oral presentation is more frequent than in 
the 1 st and 2 nd years. Thus, students in their final years are exposed to actual problems faced in the engineering 
industry and become more capable of providing solutions in the various mechanical engineering fields. 
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Figure 5. Big picture for KM from the 5 th to the 8 th semester 


4. Conclusion 

The evaluation of the prevailing curriculum used in KM courses by Big Picture Assessment was successfully 
conducted. The majority of the courses under the mechanical engineering program of UKM incorporated POl, 
P02, P05, P06, and P07 during the 1 st and 2 nd years, whereas the remaining levels were covered in the last 2 
years of the program. The students were also tested on the highest levels of the three domains of Bloom’s 
Taxonomy during their 3 rd and 4 th years. However, based on the results, most of the courses tended to use the 
cognitive domain as the primary evaluation method over the affective and psychomotor domains. Lecturers were 
also more likely to insert the element of PBL and case studies in the final year of the course. Comparisons with 
the manufacturing program are proposed for future studies to achieve a more thorough analysis. 
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